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LONG-TERM  GOALS 

Our  long-term  goal  is  to  improve  operational  atmospheric  and  oceanic  forecast  capability  by 
maximizing  forecast  accuracy  while  minimizing  observational  and  computational  costs  incurred  in 
producing  the  forecast. 

OBJECTIVES 

Our  objective  is  to  develop  and  implement  new  methods  for  enhancing  forecast  capability  on  two 
broad  fronts.  The  first  is  based  on  advancing  optimal  statistical  estimation  theory  and  developing 
practical  methods  for  applying  it  to  operational  forecast.  The  challenge  is  to  gain  the  advantage  of 
optimal  state  estimation  which  maximizes  the  accuracy  of  the  initial  state  estimate  from  which  the 
forecast  is  made  while  minimizing  the  a  priori  overwhelming  computational  burden  optimal 
estimation  methods  normally  present.  Optimal  state  estimation  is  particularly  attractive  because  it 
makes  existing  observational  resources  more  valuable  by  extracting  the  maximum  information  from 
them.  Our  second  objective  is  to  develop  and  implement  new  methods  for  optimal  ensemble 
generation  for  use  in  ensemble  forecast.  Ensemble  forecast  method  can  improve  forecast  accuracy  and 
provide  error  bounds  on  the  forecast  but  this  technique  is  sensitive  to  the  choice  of  ensemble  members. 
Our  objective  is  to  produce  optimal  ensembles. 

APPROACH 

Our  approach  to  developing  and  implementing  optimal  state  estimation  is  first  to  improve 
fundamental  understanding  of  error  dynamics  and  then  use  this  theoretical  base  to  develop  practical 
methods  for  computing  error  statistics  so  that  optimal  state  estimation  algorithms  can  be  efficiently 
implemented.  We  accomplish  this  by  reducing  the  dimension  of  the  error  system.  Having  the 
objective  of  practical  application  of  this  theory  we  are  particularly  interested  in  developing  algorithms 
for  optimal  state  estimation  that  use  forecast  products  and  algorithms  such  as  the  adjoint  integrator  that 
is  already  available  at  many  forecast  centers  where  it  is  used  for  operational  variational  data 
assimilation. 

The  theoretical  foundation  of  the  method  is  provided  by  recent  theoretical  advances  in  non-normal 
time-dependent  stability  analysis.  We  use  these  advances  together  with  methods  of  modem  control 
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theory,  specifically  balanced  Hankel  operator  truncation  methods,  to  reduce  the  dimension  of  the  error 
system.  Having  reduced  the  order  of  the  error  system  we  then  use  the  reduced  order  system  to  obtain 
an  effective  Kalman  gain  for  state  identification  in  the  full  forecast  system.  Implementing  these 
methods  in  operational  forecast  environments  also  requires  modifying  theoretically  optimal  methods 
to  use  available  resources  to  best  advantage. 

In  the  second  area  of  our  work  we  are  developing  theory  and  methods  for  incorporating  effects  of 
uncertainty  in  the  forecast  error  system.  Uncertainty  can  arise  in  the  forecast  system  both  from 
incomplete  knowledge  of  the  forecast  trajectory  and  from  statistically  based  parameterization  of 
physics.  Ensemble  forecast  is  improved  by  more  accurately  accounting  for  these  influences  on  forecast 
error  growth  which  are  of  a  different  class  than  the  more  familiar  uncertainties  associated  with  initial 
conditions.  Our  approach  is  to  apply  recent  theoretical  advances  in  non-normal  time-dependent 
stability  analysis  to  the  problem  of  error  growth  in  uncertain  forecast  systems.  Optimal  excitation 
theory  will  then  be  used  to  construct  optimal  ensembles. 

The  above  described  work  is  a  joint  effort  between  Professors  Brian  Farrell  and  Petros  Ioannou. 

WORK  COMPLETED 

We  have  completed  work  on  the  fundamental  theory  for  error  growth  in  time  dependent  systems. 

We  have  completed  the  theory  for  optimal  reduction  of  the  error  system  order  and  have  shown  that 
error  dynamics  in  these  reduced  equations  accurately  models  time  dependent  forecast  system  error. 

We  have  adapted  the  control  theory  approach  of  optimal  balanced  Hankel  operator  model  order 
reduction  previously  used  in  association  with  discrete  time  independent  engineering  systems  to  the 
time  dependent  forecast  error  dynamical  system. 

We  have  constructed  a  reduced  order  Kalman  filter  using  balanced  truncation  and  demonstrated  its 
ability  to  accurately  estimate  the  state  of  the  model  forecast. 

We  have  developed  the  theory  of  error  growth  in  uncertain  forecast  systems  and  obtained  the  error 
growth  and  structure  under  the  influence  of  statistically  distributed  parameterizations. 

RESULTS 

We  have  developed  a  theory  for  stability  in  time  dependent  systems  that  facilitates  prediction  of 
error  statistics  and  we  have  shown  that  the  dominant  error  growth  arises  intrinsically  from 
destabilization  of  a  restricted  set  of  non-normal  vectors  of  the  mean  operator  by  time  dependence. 
Together  these  results  allow  us  to  reduce  the  dimension  of  the  unstable  dynamics  of  time  dependent 
error  systems  by  representing  the  dynamics  in  a  similarly  restricted  subspace. 

We  have  developed  a  method  for  reducing  the  dimension  of  the  time  independent  error  system 
retaining  the  dominant  error  subspace.  This  method  of  optimal  order  reduction  of  the  error  system 
truncates  the  error  dynamics  in  balanced  coordinates  in  which  the  stochastic  optimals  and  the 
empirical  orthogonal  functions  collapse  to  the  same  set  of  structures  (Moore,  1981;  Glover,  1984; 
Farrell  and  Ioannou,  1996). 
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Figure  1:  Comparison  of  the  accuracy  of  3  methods  of  state  ( y/)  identification  made  using  a 
midlatitude  storm  track  model  with  400  degrees -of-  freedom  (dof)  observed  at  a  single  height  at  the 
jet  entry.  The  full  400  dof  Kalman  filter  is  theoretically  optimal  ( bottom  curve )  but  in  practice  the  3 
order  of  magnitude  error  reduction  achieved  by  the  order  60  approximate  Kalman  filter  is 
sufficiently  accurate  for  the  purpose  of  practical  forecast  ( middle  curve).  Present  methods  of  state 
estimation  corresponding  to  direct  substitution  fail  to  identify  the  state  (top  curve). 


We  have  implemented  and  tested  the  balanced  truncation  method  for  reducing  the  dimension  of  the 
forecast  error  systems  in  model  problems.  A  very  good  approximation  to  the  error  dynamics  is 
obtained  with  dimension  reduction  by  as  much  as  a  factor  of  10  (Farrell  and  Ioannou,  2001a). 

We  have  determined  that  truncating  the  dynamical  system  in  a  balanced  realization  of  the  optimals  and 
evolved  optimals  for  a  single  appropriately  chosen  time  provides  a  nearly  optimal  reduced  order  error 
system.  This  is  a  significant  result  because  it  suggests  that  implementing  the  order  reduction 
algorithm  in  an  operational  forecast  mode  can  be  greatly  simplified. 

We  have  extended  the  balanced  truncation  method  of  optimal  order  reduction  to  a  time  dependent 
Lyapunov  unstable  error  system  and  obtained  a  reduced  order  Kalman  filter  in  a  model  time 
dependent  tangent  linear  forecast  system.  We  showed  that  this  reduced  order  Kalman  filter 
successfully  observes  a  model  of  the  atmospheric  error  state  (Fig.  1)  (Farrell  and  Ioannou,  2001b). 

We  have  developed  the  theory  of  error  dynamics  in  uncertain  systems  for  application  to  the  ensemble 
forecast  problem  and  obtained  exact  dynamical  equations  for  the  evolution  of  an  ensemble  mean  field 
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and  the  evolution  of  the  ensemble  covariance  under  uncertain  dynamics  building  on  previous  results 
obtained  for  low  dimensional  systems  (Kubo,  1963;  Frisch,  1968). 


Figure  2:  An  uncertain  forecast  error  system  maps  a  single  initial  state 
to  a  distribution  of  states  at  forecast  verification  time. 


We  have  solved  the  problem  of  optimal  excitation  for  uncertain  systems.  Because  of  the  uncertainty 
the  future  states  of  the  system  are  entangled  (see  Fig.  2)  and  the  optimal  initial  state  that  produces  the 
greatest  expected  error  growth  need  not  be  produced  by  a  single  initial  condition.  We  proved  that  the 
optimal  excitation  problem  for  uncertain  systems  has  a  solution:  in  uncertain  systems  there  is  a  sure 
initial  condition  producing  the  greatest  expected  perturbation  growth  and  a  sure  structure  that  is  most 
effective  in  exciting  variance  when  this  structure  is  continuously  forced.  This  result  is  being  extended 
to  provide  a  method  for  optimal  ensemble  generation. 

IMPACT/APPLICATION 

The  methods  developed  in  this  work  can  be  directly  extended  to  operational  forecast  models.  Efficient 
calculation  of  error  statistics  using  our  results  allows  implementation  of  advanced  methods  of  state 
estimation  including  the  Kalman  filter. 

TRANSITIONS 

We  have  begun  implementation  of  the  reduced  rank  Kalman  filter  at  ECMWF. 

RELATED  PROJECTS 

None 
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SUMMARY 


We  have  made  significant  progress  toward  the  goal  of  increasing  forecast  accuracy.  A  fundamental 
theory  for  the  growth  of  errors  in  forecast  has  been  developed  and  used  together  with  control  theory 
methods  to  obtain  a  highly  effective  approximation  to  optimal  state  estimation  for  the  purpose  of 
improving  secification  of  the  forecast  initial  condition.  Our  method  for  obtaining  more  accurate  initial 
conditions  has  been  configured  to  use  operational  forecast  products  and  is  presently  being 
implemented  in  an  operational  forecast  at  ECMWF. 

We  are  developing  a  theory  for  forecast  error  dynamics  that  will  provide  methods  for  taking  account 
of  the  effect  on  forecast  error  growth  and  structure  of  uncertainty  in  the  forecast  trajectory  and  in  the 
forecast  model  parameterizations. 
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